The Challenge of Indirection: Treating Flags During Sound Analysis of Machine Code

A key problem in reverse engineering executables is to reconstruct the programs’s control flow, i.e., to construct a graph representation of the byte sequences or paths a computer may execute as instructions. Unfortunately, exactly computing all feasible paths of a program is not possible. Therefore, the task of precise control flow reconstruction and disassembly is also not solvable in general, and one must be content with an approximative answer. Contemporary disassemblers, such as objdump, radare2, and IDAPro either use linear sweep[5] or recursive descent[5] techniques, enhanced by heuristics, to compute an approximated disassembly. As a result, they may produce assembly that not only omits paths that were feasible in the analyzed binary, but also contain paths that were infeasible in the analyzed binary. While this form of approximation is acceptable for certain tasks, e.g., program understanding, it is unacceptable for tasks such as verification, where one reasons about statements that must hold on all paths (e.g., there exists no path containing an exploitable sequence). One approach to enable tasks such as verification is to allow only overapproximation, i.e., the reconstructed control flow must contain at least all paths that are feasible in the analyzed executable, but may contain additional paths. Tools that overapproximate in this way are called sound. When a verification task requires proving that an illegal program state cannot be reached, a sound disassembly can be used. If no illegal program state can be reached in the sound disassembly, no illegal program state is reachable in the original program. If it cannot be shown that no illegal program state can be reached in the sound disassembly, then it is unclear if a fault exists or not. In the following sections, we will first explain sound control flow reconstruction via value set analysis[1] (Section 2) and introduce a challenge that is unique to the analysis of machine code (Section 3). In sections 3.2 and 3.3, we will present two approaches to this problem that we implemented in our BDDStab[4] plug-in for the software analysis framework Jakstab[3]. 2 Sound Analysis of Machine Code